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DETAILED ACTION 



1. This Office action for U.S. Patent Application 10/713,130 is responsive to 
connnnunications filed 07 May 2009, in response to the Final Rejection of 18 February 
2009. Currently, Claims 11, 12, 14-17, 19-24, 35, 49-53, 55-58, 60-65, and 67-70 
are pending. 

2. In the previous Office action. Claims 52, 53, 55-58, 62, 63, 67, 68, and 70 were 
rejected under 35 U.S.C. 101 as non-statutory. Claims 11, 12, 14-17, 19, 20, 23, 24, 
35, 49-53, 55-58, 60, 61, 64, 65, and 67-70 were rejected under 35 U.S.C. 103(a) as 
obvious over U.S. Patent 6,111,604 A (Hashimoto et al.) in view of U.S. Patent 
6,148,031 A (Kato), U.S. Patent 5,987,179 A (Riek et al.), and ISO/IEC 11172 Part 1 
(MPEG-1 Part 1). Claims 21, 22, 62, and 63 were rejected under 35 U.S.C. 103(a) as 
obvious over Hashimoto et al., Kato, Riek et al., and MPEG-1, and in view of U.S. 
Patent 6,327,423 B1 (Ejima et al.). 



Response to Arguments 

3. Applicant's arguments filed with respect to the rejections of the claims under 35 
U.S.C. 101 have been fully considered but they are not persuasive. Applicant notes 
that the claims set forth the apparatuses for receiving picture signals and audio signals, 
but the claims do not state what performs the claimed processes of encoding the 
picture signals and audio signals. The Office currently interprets Bilski as requiring a 
method claim to recite explicitly in the body of the claim what apparatus or device 
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performs what the Applicant considers the inventive step in the claim to fulfill the "tied 
to" test. 

Applicant's arguments filed with respect to the rejections made under 35 U.S.C. 
103(a) have been fully considered but are unpersuasive. Applicant is reminded that one 
cannot show nonobviousness by attacking references individually where the rejections 
are based on combinations of references. See In re Keller, 642 F.2d413, 208 
USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 
1986). The previous Office action did not rely on the teachings of Hashimoto or Kato to 
disclose the multiplexing of still picture data and audio data into packs. Instead, MPEG- 
1 was relied on (Office action, pg. 8). However, at least the Riek reference discloses 
that it was known to encode still pictures in the MPEG-1 format (column 3: line 47). 
Applicant alleges in page 22 of the arguments that "nothing in Riek et al. shows, 
teaches, or suggests that the multiplexed data is composed of a first pack containing 
encoded I picture data, encoded P or B picture data and encoded audio data, and a 
second pack containing encoded audio data and encoded P or B picture data as 
claimed in claims 11, 35, 49-52, and 67-70". However, while Riek et al. does not 
explicitly discuss pack structure produced, It implicitly discloses the claimed structure by 
virtue of being an MPEG-1 encoder. MPEG-1 Part 1 was cited in the previous Office 
action to expand on the teachings of Riek in disclosing this pack structure. MPEG-1 
Part 1 shows both a pack containing I, P, and B picture data, and a pack containing P 
and B picture data only. In the example shown in § 1-A.6.5, a first pack comprises 
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pictures 1 1, 4P, 2B, 3B, and TP, 5B, and 68, and a second pacl< comprises pictures 
10P, 8B, 9B, 131, 11B, 12B, 16P, and 14B. 

Applicant additionally states in page 23 of the Arguments that MPEG-1 is 
inapplicable to the current invention since MPEG-1 "only discloses multiplexing moving 
data and audio data" whereas the present invention is directed to "multiplexing still 
picture data and audio data". However, Riek et al. clearly shows that MPEG-1 may be 
used for encoding still picture data. To bring Applicant's argument to its logical 
conclusion, the Riek et al. invention must be inoperative or incredible. In an even more 
general case, any video camera that encodes MPEG-1 video in which no moving object 
is in the field of view for a period of time, such as for example a security camera 
recording video of an unoccupied hallway or a camera in an art documentary that 
captures a shot of a painting, encodes "still picture data" in "video packs", in which the 
video packs do not record any motion between frames. With all due respect, 
Applicant's view that MPEG-1 video must capture constant or continuous motion to 
produce valid data and somehow cannot capture a motionless video is incorrect. 

Considering the above, the Examiner respectfully maintains all rejections. 

Claim Rejections - 35 USC § 101 

4. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 

5. Claims 52, 53, 55-58, 62, 63, 67, 68, and 70 are rejected under 35 U.S.C. 101 
because the claimed invention is directed to non-statutory subject matter. Supreme 
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Court precedent^ and recent Federal Circuit decisions^ indicate that a statutory 
"process" under 35 U.S.C. 101 must (1) be tied to anotlier statutory category (sucli as a 
particular apparatus), or (2) transform underlying subject matter (such as an article or 
material) to a different state or thing. While the instant claim(s) recite a series of steps 
or acts to be performed, the claims neither transform underlying subject matter nor 
positively tie to another statutory category that accomplishes the claimed method steps, 
and therefore do not qualify as a statutory process. The above claims do not recite 
what apparatus performs the claimed encoding steps. The "photographing means" and 
"audio inputting means" in claim 52, for example, are not recited as performing any 
active steps in the claimed encoding method. Contrast this with claims 61, 64, and 65, 
which all tie method steps to specific devices such as the "record medium" of claim 61 , 
the "picture decoding means" of claim 64, described as performing the claimed step of 
decoding the stored encoded picture data, or the "photographing means" of claim 65, 
described as performing the claimed step of outputting a picture signal. 

Claim Rejections - 35 USC § 103 

6. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 

forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the phor art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 



' Diamond v. Diehr, 450 U.S. 175, 184 (1981); Parker v. Flook, 437 U.S. 584, 588 n.9 (1978); Gottschalkv. Benson, 
409 U.S. 63, 70 (1972); Cochrane v. Deener, 94 US 780, 787-88 (1876). 
^ In reBilsU, 88 USPQ2d 1385 (Fed. Cir. 2008). 
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7. Claims 11, 12, 14-17, 19, 20, 23, 24, 35, 49-53, 55-58, 60, 61, 64, 65, and 67- 
70 are rejected under 35 U.S.C. 103(a) as being unpatentable over US Patent 
6,111,604 A (Hashimoto et al.) in view of US Patent 6,148,031 A (Kato), in view of US 
Patent 5,987,179 A (Risk et al.), and in view of ISO/IEC 11172-1 (MPEG-1 Part 1). 
Hashimoto et al. teaches a digital camera. 

Regarding claims 11, 35, and 52, figure 8 of Hashimoto et al. shows a block 
diagram of the camera. Image photographing section 6 comprises lens 7, lens opening 

8, imaging element 9, and filter 10. The analog input image signal is converted in 
analog/digital converter 4 and further processed in DSP 11 (column 6: lines 40-61). 
This corresponds with the claimed "photographing means". Audio signals are input into 
microphone 1 and output through amplifier/filter 2a to analog/digital converter 4 (column 
6: lines 18-26). This corresponds with the claimed "audio inputting means". Image data 
compression/expansion circuit 12 encodes the images from DSP 1 1 in a format such as 
JPEG or MPEG (column 6: line 62-column 7: line 2). This corresponds with the 
claimed "video encoding means" that performs the steps of "encoding the video signal" 
in the two encoding methods in claims 35 and 52. Figure 11 of Hashimoto et al. 
illustrates the process for capturing video and information. When the user presses the 
shutter button, a first picture with associated audio is captured. Image and audio files 
are stored in memory card 16, and an association file is written to link the image and 
audio files together (column 9: lines 46-54). The association file may be a container file 
for a still image, a still image with audio data, or a moving image with audio data 
(column 10: lines 1-8). 
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The present invention differs from Hasliimoto et al. in that in the present 
invention, two encoding methods exist: a first mode encoding only still image data, and 
a second mode encoding audio data with still picture data or moving picture data 
comprising intra I frames and inter P or B frames. However, in Hashimoto et al., no 
clear distinction is made between various encoding modes for the recorded pictures. In 
the example given In column 9: line 55-column 10: line 4, audio may be associated with 
a still JPEG image, or a moving MJPEG image, or an MPEG image (column 6: line 65), 
which was known In the art at the time the invention was made to incorporate sound 
data. 

Kato teaches an image processing system in a digital camera. Regarding claims 
1 1 and 35, In Kato, In a continuous Imaging mode, input images are Initially recorded In 
real time in an intra mode as a succession of JPEG images, and stored in memory 20 
(column 3: lines 41-47). In a still image mode, the input image is recorded In memory 
20 as a single JPEG image (column 3: lines 47-53). After recording is finished, system 
control circuit 26 re-encodes the recorded series of intra images in an Inter-frame 
compression mode (column 3: lines 54-63). This system control circuit corresponds 
with the claimed "controlling means" of claim 1 1 , and the selection of a still image mode 
or a motion image mode in Kato corresponds with the claimed recording mode selection 
In claims 35 and 52. 

Hashimoto et al. discloses a portion of the claimed invention, but not encoding 
pictures according to two different encoding methods. Kato teaches that it was known 
to encode motion image data in a separate format than still image data. Therefore, it 
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would have been obvious to one liaving ordinary sl^ill in the art at the time the invention 
was made to modify the camera of Hashimoto et al. to re-encode pictures having a 
temporal aspect, such as pictures with associated sound, as inter-frame encoded 
images after encoding, as taught by Kato, since Kato states in column 2: lines 16-34 
that such a modification would enable the final recording to be achieved with higher 
compression than with intra pictures alone, while maintaining the ability for a user to 
record a high-quality still image during the motion image recording process. 

The present invention differs from the combination of Hashimoto et al. and Kato 
in that in the present invention, a still image recorded with sound is recorded in the 
same mode containing I pictures, P pictures, or B pictures as motion images recorded 
with sound, in Hashimoto et al., a still picture recorded with sound is a single JPEG 
Image with an associated audio file. 

Riek et al. discloses a camera that encodes still images in an MPEG bitstream. 
Regarding claim 1 , figure 2 illustrates an embodiment of the Riek et al. apparatus. Light 
is input through lens 12 to CCD 14, which forms images (column 4: lines 15-18). These 
images are converted to a standard digital format in ISO CCIR601 converter 27 (column 
4: lines 35-38). As will be shown below, the images received may be encoded as still 
images or moving images. A user may switch from recording motion images to 
recording still images with still select button 22 which causes logic and control unit 32 to 
encode a still image (column 4: lines 41-50). During a still image mode, a still image 
stored in frame store 29 from converter 27 is selected for encoding (column 4: lines 41- 
46), rather than directly from the converter 27. Encoder 30 encodes a still image as a 
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series of zero-motion-vector B frames or an enlianced I frame or P frame followed by a 
series of B frames, and encoding the first frame at tlie conclusion of recording the still 
image as the next I frame (column 9: line 22-column 10: line 41). Then, Riek et al. 
discloses the claimed second encoding method which may capture both still picture 
data and moving picture data, each comprising I pictures and P pictures or B pictures. 

Hashimoto et al., in combination with Kato, discloses a majority of the claimed 
invention except for a single encoding method suitable for both still and motion image 
data. Riek et al. teaches that it was known to encode a sequence of still pictures as 
MPEG pictures integrated within a motion picture sequence. Therefore, it would have 
been obvious to one having ordinary skill in the art at the time of the invention to modify 
the encoder of Kato to record still pictures containing sound as inter pictures as with 
motion pictures containing sound, as taught by Riek et al., since Riek et al. states in 
column 3: lines 1-59 that such a modification would allow for high quality still images to 
be encoded within a motion image at a relatively low bit rate taking advantage of the 
inter-picture coding techniques of MPEG. 

However, while the present invention is directed to multiplexing an encoded 
picture signal and an encoded audio signal, Hashimoto et al. does not give details of its 
process of "combining" a video file and an audio file (column 1 1 : lines 34-42). 

MPEG-1 Part 1 defines the system coding layer of an MPEG-1 coded data 
stream, in which audio and video data streams are multiplexed (forward). Regarding 
claims 11, 35, and 52, Section 1-A.6.3 illustrates a sample multiplexing of a stream 
having one video and one audio stream. The stream is divided into packs, each of 
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which has a header and three pacl^ets, each of 2048 bytes. Section 1-A.6.5 illustrates 
the distribution of frames into the packs, with a first pack typically encoding I, P, and B 
picture data and a second pack typically encoding P and B picture data in conventional 
GOP structure and video of average complexity. First, 13 video packets are transmitted 
to ensure successful buffering. Then, an audio packet is placed for every 6.25 video 
packets. Section 1-A.6.9 shows an extended sample multiplexed data stream. Here, a 
second audio packet is placed between the twentieth and twenty-first video packets. 
However, while in the shown example, one audio packet is placed for multiple video 
packets, the examiner takes Official Notice that it was known in the art for audio and 
video packets to be correlated in a 1 :1 ratio, as in the "locked audio" of DV, in which one 
audio pack is present for each frame. Such a modification would ease linear editing to 
prevent mismatches between audio and video data streams at a start point or an end 
point of an edited segment. 

Hashimoto et al., in combination with Kato and Riek et al., discloses the claimed 
invention except for multiplexing an audio and picture signal. MPEG-1 Part 1 teaches 
that it was known to produce a multimedia datastream by multiplexing packets of audio 
and video data. Therefore, it would have been obvious to one having ordinary skill in 
the art at the time the invention was made to implement the combination of a video and 
audio file in Hashimoto et al. as a multiplexing operation, as taught by MPEG-1 Part 1, 
since MPEG-1 Part 1 states in the Introduction that such a modification would allow for 
synchronized playback of audio and video data without having to buffer an entire sub- 
stream. 
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Regarding claims 12 and 53, in Kato, "tine JPEG standard is used in tine still 
image compression and the MPEG standard is used in the moving image compression" 
(column 9: lines 12-14). 

Regarding claims 14 and 55, figure 12 of Hashimoto et al. shows video files and 
audio files stored In separate areas of memory card 16. 

Regarding claims 15 and 56, in Kato, as mentioned previously, video data is first 
stored in first memory 20 and then transferred to second memory 22 (column 3: lines 
54-63). This corresponds with writing multiplexed data to memory, reading the 
multiplexed data from memory, and recording multiplexed data on a recording medium. 
Additionally, in Kato, during the recording of a moving image, a still image from the 
sequence of moving images may be additionally transferred from the first memory to the 
second memory as an intra picture in an independent process of the moving picture 
recording (column 4: lines 1-10). This corresponds with encoding a video signal in the 
"first" encoding method, writing the signal to the memory, reading the signal from the 
memory, and recording the signal to the recording medium. 

Regarding claims 16 and 57, figure 5 of Kato shows a compressed video 
encoder including DCT circuit 107, quantizing circuit 108, and variable length coding 
circuit 115 (column 5: line 60-column 6: line 24). 

Regarding claims 17 and 58, figure 12 of Hashimoto et al. illustrates audio and 
video files stored in the memory as having headers. 
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Regarding claims 19 and 60, an MPEG-1 pack, containing 3 packets, is designed 
to have a pack rate of 29 Hz, or 1 frame per pack (Section 1-A.6.3). 

Regarding claims 20 and 61, Kato temporarily stores incoming data on first 
memory 20, and after re-encoding, permanently stores the data on second memory 22 
(column 4: lines 45-55, "the second memory 22 is the final storage medium"). 

Regarding claims 23 and 64, in Hashimoto et al., image data compression circuit 
12 may also perform image decoding (column 6: lines 62-66), and so corresponds with 
the claimed "video decoding means". The decoded video signals may be further 
processed in DSP 11 (column 6: lines 58-61), which outputs a video signal 26 to a 
display such as an LCD viewfinder (not shown in figure 8). This display corresponds 
with the claimed "displaying means". Digital audio signals may also be decoded in 
audio data compression/expansion circuit, transmitted to D/A converter, amplified and 
filtered in amplifier 2b, and output in output stream 26 to speaker 32 (column 5: lines 17- 
39). This corresponds with the claimed "audio outputting means". This process of 
reading data stored in memory card 16 (column 7: lines 34-50), like all other processes 
of the camera of Hashimoto et al., is controlled by CPU 23 (column 7: lines 15-16), 
which corresponds with the claimed "controlling means". 

Regarding claims 24 and 65, the CCD in Hashimoto et al. has a resolution of 768 
X 480 pixels (column 6: line 44), and Kato inputs images at a resolution of 720 x 480 
pixels, in accordance with the NTSC standard (column 4: line 15), and produces an 
output of 320 X 240 pixels (column 4: line 24), in accordance with the GIF format. 
Although neither Hashimoto et al. nor Kato et al. record pictures at the VGA 640 x 480 
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pixel standard, it would have been an obvious matter of design choice to modify the 
image sensing portion of camera of Hashimoto et al. or of Kato to produce 640 x 480 
pictures, since it has been held that a change in size of a component is generally 
recognized as being within the level of ordinary skill in the art. See In re Rose, 105 
USPQ 237 (CCPA1955). 

Regarding claim 49, in Kato, as shown in figure 1, system control circuit 26 
controls both the image compression circuit 18, first memory 20, and second memory 
22 (column 3: lines 41-63). This system control circuit corresponds with the claimed 
"controlling means". 

Regarding claims 50 and 67, in Hashimoto et al., incoming image data from a 
camera is processed in noise reduction circuit 10 and DSP 11 (column 6: lines 40-61), 
and incoming audio data from a microphone is processed in amplifier/filter 3a (column 
6: lines 18-21). 

Regarding claims 51 and 68, in Hashimoto et al., figure 14 illustrates the 
flowchart for transmitting and receiving data from the camera to an external device 
(column 10: line 41-column 11: line 42). Data from the memory card is transferred to 
FIFO 13 (column 11: lines 25-29), and transmitted to an external device via interface 
circuit 27 (column 7: lines 1-36). Like every other process in the camera of Hashimoto 
et al., this process is controlled by CPU 23 (column 7: lines 15-16), which corresponds 
with the claimed "controlling means". 

Regarding claims 69 and 70, in Kato, digital signal processor circuit 14 
corresponds with the claimed "first picture encoder" that performs the step of receiving a 



Application/Control Number: 1 0/71 3,1 30 Page 1 4 

Art Unit: 2621 

picture signal. In Hashimoto et al., audio data compression circuit 3 corresponds with 
the claimed "second encoder" that performs the step of receiving an audio signal. In 
Kato, image compression circuit 18, as modified by Riek et al., corresponds with the 
claimed "picture generation" that performs the claimed step of "generating fixed data". 
In Hashimoto et al., FIFO 13, which combines audio files and image files (column 11: 
lines 43-61), corresponds with the claimed "third encoder" that performs the claimed 
step of "multiplexing". 

8. Claims 21, 22, 62, and 63 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Hashimoto et al. in view of Kato, Riek et al., and MPEG-1 as applied 
to claims 1 1 and 52 above, and further in view of US Patent 6,327,423 B1 (Ejima et al.). 
Claims 21, 22, 62, and 63 are directed to specific operations of causing a camera to 
perform an audio capture for a certain time. Hashimoto et al. teaches taking pictures 
when a shutter button is pressed (column 7: lines 20-24), Kato teaches operating a 
keyboard to issue image taking commands (column 3: lines 41-53), and Riek et al. 
teaches recording a still image while a still select button is depressed (column 4: lines 
41-50). However, the above references do not teach operation for a time period to 
encode audio data. 

Ejima et al. teaches a camcorder that records sound data. Regarding claims 21 
and 62, figure 14 is a flowchart illustrating one embodiment of the sound recording 
control process of Ejima et al. At step SI, CPU 39 determines if a release switch 10 is 
pressed, and if it is, the image recording process begins at step S2 (column 15: line 64- 
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column 16: line 3). At step S3, the sound recording process is started, and at step S4, a 
"REC" display is shown on a viewfinder to indicate that sound is being recorded (column 
16: lines 4-11). At step S5, after 10 seconds have passed, the sound recording process 
stops (column 16: lines 11-16, 34-40). However, if a sound recording switch is pressed 
within 10 seconds at step S6, sound recording continues (column 16: lines 14-23, 44- 
50). The sound recording then ends when the sound recording switch is released at 
step S20 (column 16: lines 23-50). Then, sound recording switch 12 corresponds with 
the claimed "operating means", and the time period in which the sound recording switch 
is pressed corresponds with the claimed "timing means". 

Hashimoto et al., in combination with Kato, Riek et al., and MPEG-1, discloses 
the claimed invention except for encoding audio during the pressing of an operation 
means. Ejima et al. teaches that it was known to perform sound recording while a 
sound recording switch is pressed. Therefore, it would have been obvious to one 
having ordinary skill in the art at the time the invention was made to incorporate the 
sound recording switch of Ejima et al. into the camera of Hashimoto et al. or Kato, since 
Ejima et al. states in column 1 : line 60-column 2: line 20 that such a modification would 
allow the timing of a sound recording to be independent of the timing of its associated 
video recording. 

Regarding claims 22 and 63, in Kato, if sound recording switch 12 is not pressed, 
then release switch 10 corresponds with the claimed "operating means", and the ten 
seconds is the "predetermined time period" in which audio is encoded. 
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Conclusion 

9. THIS ACTION IS MADE FINAL. Applicant is reminded of tine extension of time 
policy as set forth in 37 CFR 1 .136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to David N. Werner whose telephone number is (571)272- 
9662. The examiner can normally be reached on Monday-Friday from 10:00-6:30. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Mehrdad Dastouri can be reached on (571) 272-7418. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 
273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retheval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
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Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



ID. N. W./ 

Examiner, Art Unit 2621 
/Dave Czekaj/ 

Primary Examiner, Art Unit 2621 



